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To determine the relati-onship between 16 background 
variables and students* evaluations of instruction, a questionnaire 
was coppleted in 511 undergraduate courses at the University of 
Southern California, Los Angeles. Student variables, including grade 
point avera.ge, class size, expected grade, and prior subject 
interest, rarely explaiiyed 10* of the variance in any student ratings 
and generally explained less than 5%. Different statistical 
techniques, however, suggested that 12X to UX of th*» variance in the 
student ratings could be predicted by the set of ta .ground 
variables. The variables aost important in predicti evaluations 
were prior subject interest, expected grade, workload/difficulty, and 
perhaps, percent taking course for interest oni^. Of these, prior 
subject interest was the iQSt i«portant and was better interpreted as 
a variable affecting quality of education. Background variables aid 
have a saall relationship to the evaluations, but results argue 
against bias interpretation. Horkload/dif f iculty was correlated in 

• the op.posite^dirjectiqn as would be expected fro« a biaSi effect. No 
single variable was related to a' lajority of the evaluation sc^ -es. 
The scores aost likely to be biased (overall rating and instructor 
enthusiasB) were not the scorfes «ost related to the background 
variables. A sample suBBary. of the instructor's rating, , and the 

♦ questionnaire are appended. (Author/JAG) 
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ABSTRACT ' r ' . 

The purpose "of this study was to determine the relationship between 16 
background variables (GPA, Class Size, txp«?cted Grade, Prior Subject Interest, 
etc.) and students' evaluations of instruction in Gil undergraduate courses. 
Individual background variables rarely explained even 10% of the variance in 
any student ratings and generally explained less than 5%. Little or no non- 
linearity was found. Different statistical techniques, multiple regression 
and canonical correlation, suggested-that 12% to J 4Lo.tJthe. variance in, the 
student ratings could be" predicted by the set of background variables. Three, 
or perhaps four, background variables were most important in predicting students' 
evaluations: Prior Subject Interest, Expected Grade, Workload/Difficulty, and, 
perhaps. Percent Taking Course for General Interest Only. Of these. Prior 
Subject Interest was the most important. While the background variables did 
have a small relationship to the evaluations, a host of considerations argues ' 
against a simple bias interpretation. Workload/Difficulty was correlated in 
the opposite direction as would be expected with a bias effect. Also, ft 
Subject Interest was better interpreted as a variable impacting quality of 
education. Furthermore, no one background variable was^related to even a 
majority of .the evaluation scores, the effect of the background variables 

9 

varied dramatically for different evaluation ^scores, and those evaluation 
scores most likely to be subject to bias (Overall Instructor Rating and 
Instructor Enthusiasm) were not the ones most related to the background 
variables. Only Expected Grade could reasonably be considered a bias, and 
even this interpretation was subject to alternative explanations. ^ . 
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In spite of the widespread use of students' evaluations as one measure 
of effective teaching,- there is often the fear or suspicion that the ratings 
lack validity and are adversely affected by variables unrelated to the quality 
of instruction. The harshest critics even suggest that an instructor need 
only give high grades and demand little work of students in order to receive 
high evaluatigns. The purpose of this paper is to investigate the relation- 
ship between different dimensions of students' evaluations and a set of back- 
ground variables characterising the student, the course, and the instructor. 

McKeachie (1973), after reviewing a broad spectrum of student evailuation 
literature, concluded that a number of potential sources of bias apparently 
„^re of little Consequence. Remmers (1963), describing a quarter of a century 
of research with the Purdue Rating Scale for Instructors, also concluded that 
the ratings are little affected by student/course/instrucoor characteristics. 
Hildebrand, Wilson and Dienst (1971) found no correlation between students*- 
evaluations and 10 student/^urse/ instructor characteristics. Menges (Menges, 
1973; Costin, Greenough and Menges, 1971) also suggested that potential sources 
of bias have little effect on ratings, but indicated the need for further study 
of expected grades, prior student interest in the subject, class size, and 
reason f6r taking the course. Overall, Marsh and Kesler (1977) reported curvi- 
linear relationships between class size and several evaluation dimensions, but 

« 

found the effect to be large for only a Group Interaction dimension. Marsh, 
Overall and Thomas (1976) found that both expected grade and prior interest in 
the subject correlated with students' evaluations, but other background variables 
generally did not. In extensive literature reviews on the effect of expected 
grades (Marsh, Overall and Thomas,' 1976; Feldman, 1976), it was concluded that 
there is generally a small correlation between exp.ectad ;jrades and students' 
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evaluations, but that a bias in the ratings was only one possible explanation. 
Feldman (1976) also reported that interest in the subject was correlated with, 
ratings and may explain some of the relationship tetween expected grades and 
evaluations; more interest in the subject leads to higher grades and better 
teaching. In summary, no one background variable was found to have a consis- 
tently strong relationship with students' evaluations, but several— particularly 
Expected Grades, Prior Subject Interest, Class Size and, perhaps. Reason for 
Taking the Course—were found to haveleiall to moderate correlations in. a 
number of different studies. 

Fewer studies have looked at the combined effect of an entire set of 
background variables on one or more student evaluation i terns An important 
problem in this approach is a. careful determination of what are appropriate 
background variables and what are really evalu^ition items. For example. Price 
and Magoon (1971) reported that a set of 11 background variables explained 
over 20% of the variance in 24 evaluation items. However , "student ratings of 
"availability of the instructor," "explicitness of course policies," and "class- 
room atmosphere" (relaxed versus tense) were considered as background variables 
Snd contributed to the prediction of the evaluation* items. Most researchers 
would consider these variables to be part of the evaluation of teaching. 
Similarly, Pohlmann (1975) found that 9 background variables explained over 
20% of the variance in 5 rating items; however, course 'difficulty was the 
rating ijtem best predicted, and it was correlated to a conceptually-similar 
item concerning the hours spent outside of class. Brown (1974) reported that 
n background variables explained 14% of the variance in an average of student 
evaluation items, but indicated that average grade accounted for the most 
variance. Burton (1975) showed that 8 background variables (including GPA,"* 
Expected Grade, C-lass Standing, Reason for Enrolling, and Enthusiasm toward 
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the Subject) explaified between 8% and 15% of the variance in instructor ratings 
over a seven-semester period of time, but indicated that the most important 
variable was student enthusiasm toward the subjects 

In summary, studies considering the combined effect of an entire set of 
background variables generally found that the set explained at least 10% of 
the variance in students' evaluations, and some suggest that the proportion 
was as high as 25%. However, particularly with .those studies finding the 
higher estimates, there was a problem in determining what were background 

variables and what were evaluation items: either eval uation-like item s were 

included in the set of background items, or conceptual ly-similar items (e.g.. 
Hours Required and Course Difficulty) were included in both sets. 

A host of philosophical and methodological considerations complicates 
the analysis of the relationship between students' evaluations and student/ 
course/instructor variables. First, correlations cannot be used to prove 
causation. If poor tea,qhers>^*ere assigned to teach large introductory courses, 
their lower ratings should not be attributed to a bias produced by class size 
or course level. Second, the distinction between practical and statistical 
significance needs to be drawn. A statistically significant relationship based * 
upon a very large sample size may be so small as to be of no practical importance. 
Third,, the existence of curvil inear-relat-ionsirips needs to-be explored. Fourth, 
the multivariate nature of students' evaluations requires that different evalua- 
tion dimensions be considered separately; class size is moderately correlated 
with Group Interaction but shows little relationship to other dimensions. 
Fifth, the combined effect of an entire set of background variables need's to 
be determined as well as the effect of each separately. FiJially, the nature 
of the "bias" being considered needs careful attention. On the one hand, if 
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teachers need only give high grades and demand little work to receive high 
evaluations, then the evaluations' are'clearly biased. On the other hand, if 

o 

students start a class with a strong interest in the subject, they may rate 
th'w teacheV more favorably because he really was more effective than he would 
have been with less motivated students^ I" summary • the complications make 
the problem interesting, but virtually eliminate the possibility of reaching 
any definitive conclusions. 

METHOD ^- • ' L_ _ 

Evaluation Instrument " : " 

' V 

The evaluation instrument (see Appendix I) consisted of 35 evaluation 
items which define 9 different evaluation factors, and 6 additional items which 
measure background variables. Both the individual evaluation items and the 
evaluation factors are quite reliable (see Appendix II). Coefficient alphas 
CNie, et al., 1977) for the evaluation factors varied between .88 and .97. 
Factor analysis -(-see Appendix III) has supported the existence of the nine 
evaluation factors in each of three different semesters. The oblique factor 
solution resulted in factors which had low to moderate correlations varying 
between r=-.02 to r=+.49 (median r=,27). Furthermore, essentially the same 
evaluation factors were found with faculty self-evaluations of their own 
teaching when using the same instrument (Marsh and Overall, 1978). 

Students' evaluations were summarized by eleven evaluation scores, the nine 
evaluation factors and the tv^o overall summary items. Factor scores were 
weighted averages of the evaluation items, while the two overall ratings 
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were basad upon responses to single interns. The evaluation scores and brief 

descriptions are as follows: 

Learning/Value— The extent to which students felt they encountered 
a vdluable learning experience that was intellectually challenging. 

In structor, Enthusiasm— The extent to which students perceived the 
— instructor to display enthusiasm, energy, humor and an ability to 
hold interest. 

Organization— The instructor's organization of the course, course 
materials"," and class presentations. 

o' Group Interaction— Students' perceptions of the degree to which the 
instructor encouraged class discussions and invited students to share 
their own ideas or be critical of those presented by the instructor. 

individual Rapport— The extent to which students perceived the in- 
structor to be friendly, interested in students, and accessible in or. 
out of class. 

Breadth of Coverage— The extent to which students perceived the in- 
structor to present alternative approaches to the subject, and to 
emphasize analytic ability and conceptual understanding. 

■ Examinations— students' perceptions of the value and fairness of 
graded materials in the course. 

Assignments— The value o,f class assignments (readings, homework, etc.) 
in adding appreciation and understanding to the subject. 
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, ' Background Characleri sties 

' Morki o'ad/Di f f i cul ty— Students ' perceptions of the relative 
dffficulty/workload of th^^ course and the pace of presentations. 

Werall Course— A single item asking students to compare the .course 
with other courses at^USC, 

Overall Instructor— A' single item asking students to compare the 
instructor with othfer instructors at USC. 

B'ackgroifn"d~Vgr1ab1 es ^ -v* 



' The set of background variables consisted of 16 different variables 
describing the course, students in the course, and the instructor. Selection 
of this set of variables was prompted by a review of the literature and the 
availability of information. The Workload/Difficulty variable has been in- 
cluded as both an evaluation score and a background variable., However, when- 
ever the combined effect of background variables was being determined. Work- 
load/Difficulty was considered a background variable. 

Two subsets of the background "variables were given special attention in 
some of the analyses. The first subset was the Reason for Taking the Course. 
Students selected one of five possible reasons for taking the course, or left 
the item blank if none of the given reasons, were appropriate. The percentages 
ofostudents indicating each of the reasons wer^included as five separate 
varial)les. However, these five Variables shduld be interpreted cautiously, 
sihce a higher percentage for any one necessitat;?s a lower percentage on an- 
other. The second subset consisted of four highly correlated variables which 
destribe the Class Level. , 
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' The 16 background variables and brief description's are as follows: ^. 

Level of Interest in the Subject Prior to This Course (1 -Very Lqw.-v.* 
3-Medium...5-Very High)— Mean class' average response was'3.4. 

.- Work! oad/Di f f i cul ty— An evaluation 1*actor score representing four items;. 
" high vjilues refer to courses^hich are more difficult, have a heavier .. 
workload, are-faster' paced , and require a greater number of hours outside 
of class. 

— - Overa-l-l-GPA- (1 -Below 2.5, 2-2.5 to 3.0, 3-3.0 to 3.4, 4-3.4*to 3.7, 
• 5-Above^3.7)-^Mean class average response was 3.3 (i.e., slightly higher 

than a 8 average. ' . 

J Enrollment— The number of students who were enrolled in the course (mean 

enrollment was 34:5). 

Teacher Rank (1-Teaching .Assistant, 2-Lecturer, 3-Instructor, 4-Assistant 
Professor, 5-Associate Professor, 6-Full Prof essor)-Cl ass average response 
waf4.3 (Note: Teaching Assistants were excluded from the analysis, arid 
very few teachers were either Lecturers or Instructors). 
Percent Students Majoring in Same Division as the Course (e.g., %, Social 
Science students in Social Science course's) -Mean class average response 
was 49%. . 

Expected Grade (1-F,- 2-D, 3-C, 4-B, 5-A)— Mean class average response 
was^4.2 (i.e., slightly higher than a C' average). 
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i^ Reason for Taking the Course Ca subset of "five variables): 

% Indicating Major Requirement— Me^n class average response was ^2%. 

% indicating Major Elective- -Mean class average response was 2^%. 

% indicating General Interest On1y- -Mean class average response was *16%, 

% indicating General Education Requirement— Mean class average response 

was 12%. a * 

% indicating Minor/Related Field— Mean class average response^as 5%. - 

Class Level (a subset of four variables): 
Mean Year in School Q -Freshman, 2-Sophomore, 3-Junior, 4-Senior, 5-Graduate) 
Mean class average response was 3. '2. ' 

% indicating Freshman or Sophomo re— Mean class average was 25%., 
% indicating Junior or Senior— Mean class average response was 65%. 
Course Level (1 -Lower Division, 2-Upper Division)— Mean class average 
response was 1.7. • , , " 

Statistical Analysis 

All analyses were performed on class "average responses for the 511 courses 
in the study. Each of the 16 background variables was correlated with each of 
the 11 student evaluation scores. These linear relationships were considered 
substantia^ only if they predicted at least 5% of the variance in one of the 
evaluation scores (i.e., the correlation weis at 'least r=+.23). Second order 
Cquadratic) and third order (cubic) components of each background variable 



Students were also given the option of^^ving this item blank; thus, thera is 
an implicit sixth possible response category of "Other" which keeps this set 
from being completely dependent. 
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were then tested to determine i\f any substantial non-linearity existed, a rela- 
tionship Being considered substantially non-1 inRar if it accounted for at least 
b% ^f the variance in an evaluation score and if. a nonrlinear component added 
at least }% to the variance explained by the linear rela1:ionship. 

Step-wise multiple regression (Nie^ et aK, 1^75) was used to determine 
the combined effect of the background* varic^bles on each evaluation score. At , 
each^step-, the single variable which added most to the "variance explained" was. 
added to- the regression equation until no additiorjal varia^)le Could add an ' . 
additional' 1% to the^total variance already explained, ftt each step, the total 
variance .e3cpl a ined. was adjusted for the number of variajbles in the equation 
j^^Jie, et al,, '1975; CoSen and Cohen, 1975).. Cohen and Cohen. (1975) suggested 
that al"l variance estimates be corrected-for the totaTnumber of variables which- 
are available to be used in the regression equatiorf, rather than lust- the number 
used at each stepf this adjustment procedure was the\ basjis of final variance - 
estimates. . ^ - 

The proportion of variance which was uniquely contributed by ^ach of the 
background variables^^-s then^btained by determining the proportion of variance 
which could be predic jd by all but one of the background variables and then 
computing* the proportion of additional variance which could be explained by the 
one remaining variable, A variation of. this procedure wa^s used for the two 
subsets of background variables discussed earlier (Reason for Taking Course and 
Class Level). The multi-col inearity (Cohen and Cohen, 1975) dictated that any 
one variable in each set would make little contribution once the remaining • 
.variables had been included. To avoid this problem, two alternatives were con- 
sidered. First, the additional variance- contributed by the entire subset of 
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variables was determined. Second, the additional variance contributed by each 
variable in the subset was determined without including any of the other 
variables in the subset. Once again, the proportion_of ^variance explained was_ 
adjusted for the number of variables included at each step. 

Canonical correlation (Dixon, 1975; Cooley and Lohnes, 1971, 1976) was. 

e 

used to determine the relationship be,tween the entire set of background variables 
and the ^entire set of student evaluation scores. In the first step, this pro- 
cedure determines a linear combination of background variables which islnaximally 
correlated with a linear combination of student evaluation scores. At each 
successive step, additional linear combinations of variables, the canonical 
. yariates, are 'Extracted which are uncorrelated with previous ones and maxi- 
.mally correlated with each^other. This procedure is intuitively an extension 
of multiple regression in which a linear combination of background variables 
.was determined which maximally correlated with just one evaluation score. 
Cooley and Lohnes (1971 , 1976) have discussed a measure of jthe redundancy 

between two sets of variables used in canonical correlation. This measure is 

— - / 

a quantitative description of the total proportion of variance in one set of ^ 
variables ^which can be predicted by another. In this study, the>edundancy 
measure was used. to determine the proportion of variance in the entire set of 
Evaluation scores whith can be predicted by the entire set of background vari- 
a'ble's. 
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RESULTS AND DISCUSSION 

Bivariate Relationships--Linear and Non-Linear 

Linear correlations between each of the 16 background variables and the 11 
evaluation scores are presented in Table 1. Correlations as small as r=.09 
were statistically significant, though of little practical significance. Con- 
sequently, attention was focused upon those relationships which accounted for 
at least 555 of the variance in any one of the evaluation scores (i,.e., correla- 
tions, of at least r=.23). Of the 175 correlations, only 18 met this criteria, 
and only 3' of the 18 'accounted for as much as 10% of the variance; Prior 
Subject Interest was positively correlated with both Overall Course Rating and 
Learning/Value evaluation scores, and Enrollment was negatively correlated with 
quality of Group Interaction. None of the 16 background variables accounted 
for more than 5% of the variance in even a majority of the evaluation scores, 
and only three, (Prior Subject Interest, Workload/Difficulty, and Expected Grade) 
did so for more than one evaluation score. 

^ ' INSERT TABLE 1 ABOUT HERE 

s • 

4 

The extent of non-linearity was considered in each of the background- 
evaluation relationships. Only 7 of 175 relationships showed any substantial 
non-linearity. Course enrollment generally showed a non-linear relationship to 
evaluations: courses with large enrollments and small enrollments were rated 
more favorably, but only 2 of the 11 relationships reaphed'the criteria of 
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"/substantial." For the other nine evaluation scores, even with the additional 
variance accounted for by the non-linear components of Enrollment, the total 
variance explained was less than 5%, The majority of the evaluation scores 
(Overall Course, Overall Instructor, Enthusiasm, Organization, Individual 
Rapport, and Examinations) showed no substantial non-ltnear. relationship _ . 
with any of the background variables. 

In sjummary: 

**Indivikual background variables rarely accounted for as much as 10% of the 

variance in any of the evaluatipn scores -(3 of 175 relationships) and 
c generally did not even account for S% (18 of 175) • 

**More favorable evaluations tended to be given to classes in which students 
had higher Prior Subject interest and Expected Grades and those in which 
they experienced higher levels of Workload/Difficulty. 

♦♦Background \te?riables generally showed little or no non-linear relationship 
to any of the evaluation scores. 

Multivariate Relalionships— Multiple Regression 

Each of the set of 16 background variables was entered Into a step^wise 
multiple regression to predict, each of the 11 student evaluation scores. This 
analysis had two purposes: to determine the combined effect of all the back- 
ground variables on each evaluation score, and to determine which of the back- 
ground variables consistently made the largest contribution. In order to 
simplify the Interpretations, a rather conservative criterion was used to 
determine whether additional variables would be entered into the equation; an 
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additional variable was included only if it added at least 1% to the variance 

which hafl been. accounted for already by the previous set of variables. 

The percentage of variance which could be explained in each of the 

different Evaluation scores Corrected for the number of background variables 

-available) varied~dramaticaVl-y,~rang1ng fro^ 0% for Organization to 25% for 
« 

Learning/Value. The set of background variables accounted for 20% or more of 
the variance in three evaluation scores: Overall Course (20%), Group Inter- 
action (23%h and Learning/Value (25%). Four of the set of background vari- 
ables consistently appeared in the final regression equations: Prior Subject 
Interest, Expected CouVse Grade, Workload/Difficulty, and percentage of 
students indicating "General Interest" as their reason for enrolling in the 
course (as opposed to Major Requirement, Major Elective, or General Education 
Requirement). In each case, courses tended to be rated more favorably when 
Prior Subject Interest*was higher, when Workload/Difficulty was greater, when 
Expected Grades were higher, and when percent enrol lii^g for "General Interest" 
was higher. 

INSERT TABLE 2 ABOUT HERE " 

Multiple regression was also used to determine the unique contribution 
of each of the individual background variables to each of the evaluation scores 
(see Table 3). The unique variance is the proportion of additional variance 
accounted for by each variable or each subset of variables, after all other 
variables have been considered. Inspection of Table 3 indicates that much of 
the variance acc^^inted for by any one background variable is redundant with 
variance accounted for by others; although 18 of 175 relationships between 
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background variables and evaluation scores accounted for 5% of the variance, 
only 7 relationsKips uniquely accounted for at least 555 ti^e.-, variance which 
was not also explained by other background variables). Only four Background 
variables uniquely accounted for as much as 5%. of the variance in any of the 
evaluation scores: Prior Subject Interest, Workload/Difficulty, Expected- 
Grade, and Reason For Taking Course. 

IN§iE!.IABLE_3_AB0yT_HERE 

An interesting relationship consistently appeared between Expected Grade, 
Workload/Difficulty, and the evaluation scores- Expected Grade an.d Workload/ 
Difficulty both tended to be positively related to each of the evaluation scores, 
but were negatively related to each other (r=-,29). This rather unusual event 
Is a case of cooperative or reciprocal suppression which is described by Cohen 
and Cohen 0975)"? While this occurrence is interesting, it also complicates 
interpretations. The combined effect of the two variables is necessarily 
greater than the sum of their individual effects. Furthermore, the supposedly 
"unique" variation attributable to either variable may be greater than variance 
explained before the effect of other variables has been removed. 

In summary: 

♦♦Percentage of variance in different evaluation scores which was explained by 
the set of background variables varied dramatically, ranging from 0% to 25%; 
the average was 11. 8%. 

♦♦Background variables explained 20% or more of the variance in 3 evaluation 
scores: Learning/Value, Group Interaction, and Overall Course. 
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**Only four of the background variables cons.istent.ly appeared in the final 
regression equations: Prior Subject Interest, Expected Grade, Workload/ 
Difficylty, and percentage indicating "General Interest Ohly" as the 
reason for taking the course. ' , . ^ 

**Mach of the yariartte predicted by individual background variables was 
redundant with variance predicted by other background variabl-es. 

Multivariate Relationships— Canonical Correla tion " , 

' Canonical correlation is a gencra-l statistical technique for determining ■ 
the combined effect of one set of variables (the background variables) on an- 
other set of variables (the evaluation scores).. When only one evaluation score 
is considered, the tecKnique is equivalent tp multiple regression. When more 
than one evaluation score is considered, linear combinations of the background 
variables and the evaluation scores are determined - so that their correlation 
is maximal. On successive steps in the analysis, additional pairs of linear 
combinations are extracted which are again maximally correlated with each other 
and uncorrelated with previous linear combinations. The magnitude of the 
canonical correlations can easily be misinterpreted. For example, if the same 
variable were included in both sets, the first canonical correlation would 
necessarily by r=1.0--a perfect relationship. Each canonical varlate would 
consist of only the one variable in common to the two sets, and all other 
variables would have a zero weighting. Cooley and Lohnes (1971, 1975) describe 
a redundancy measure which indicates how much variance in one set of variables 
can be predicted by a second set of variables. If Workload/Difficulty were 
included as both a background variable and an evaluation score, the first 
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canonical correlation would be r=1.0, but the redundancy measure would be 
only 955 Ce.g., 1 of the 11 evaluation scores would be perfectly, predicted, 
but the Ot'fier 10 would not be predicted at all). 

Inspection of Table 4 indicates that 9.5% of the variance in the evalua- 
tion scores can be explained by the first pair of canonical variates. 
Successive canonical variates account for 2.3%, 0.8%,. 3.2% and 0,5% of the 
variance, in. the evaluation scores. The sum of these values, appr6)^imately ' 
16%, is an estimate of the variance in the entire set of students' evaluations 

which can be explained by the entire set of background variables. This 

' 2 
sample estimate is -inflated in the same way as the multiple R in multiple 

regression, but no adjustment procedure to correct for this positive bias has 

been developed. If this variance estimate were' corrected for the use of 16 

background variables' in the same manner as the multiple (Coheri and Cohen, 

1975), the corrected estimate would be 13.6%. This result, of course, 

corresponds rather closely to the 11.8% value which was based upon the 

successive multiple regression ~ equations. Furthermore^ Prior Subject laterest 

Workload/Difficulty, and Expected Grade were again found to be the most im- 

portarit in predicting the evaluation scores; while Learning/Value, Group inter 

action, and Overall Course Rating wer-is the evaluation scores which were best 

predicted. However, teacher Rank was also shown to be important in this 

particular analysis; higher ranked teachers (e.g.,, Full Professors) were 

rated as giving broader coverage, better assignments, and poorer Group Inter- 

action, 

, ^ 

INSERT TABLE 4 ABOUT HERE 
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In- summary: 

♦♦Canonical correlation indicated that 1655 of the variance in the entire set 
of tvaluatTon scores can be explained by the entire set of -background 
variables. 

♦♦Background variables^ most important in explaining evaluation scores were 
Prior Subject Interest, Workload/Difficulty, Expected Grade, and Teacher Rank. 

♦^Evaluation scores which were best predicted were Learning/Value, Group Inter- 
acti^-, and Overall Course Rating. 

Most important Background Variables and Explanatory Models 

A difficult problem in multivariate research is the search for "the" most 
important independent variables (the background variables)'' used in predicting 
one or more dependent variables (the evaluation scores)'.' Many alternative-, 
critferia, including their strengths and weaknesses, have been discussed else- 
where, but the most comnonly suggested are: 1) simple correlations, 2) Beta ' 
weights in final' regression equations, 3) the change in variance explained as 
each new variable enters a regression equation, and 4) the variance which is 
uniquely defined by each independent variable. Fortunately, in the present - 
situation, each of these criteria suggested that the same three, or perhaps 
four, background variables weVe most important: Pjfior Subject Interest, Work- 
load/Difficulty, Expected Grade, and perhaps Reason for Taking the Course. 
Most favorably rated courses tended to have students who were more interested 
in the subject before the start of the. course, tended to have students who 
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expected to receive higher grades, tended to have a heavier workload and be more 

% ? 

difficult, and tended to hav,e more students taking the course for general 
Interest. only. * - q 

"Path "analysis, commonality modeling, and related techniques of variance 
partitioning CCooley and Lohnes, 1976; Blalock, 1971) attempt to determine the 
relat|ye contribution of each independent variable and how it affects the 
dependent variables. While none of these techniques allow the researcher to 
draw causal conclusions on the basis of correlational dati, they do providte a 
systematic approach to testing some causal hypotheses- Generally, the first 
step is to establish the temporal ordering of .the independent (background) 
■variables, and then to determine the proportion of variance directly attribu- 
' table to eachJ For example, Fel-dman (1976) speculated that the observed rela- 
tionship between grades and evaluations may be partially or fully due to the 
fact that both are causa-lly related to some antecedent variable such as prjor 
interest. He proposed a path analysis which would test this hypothesis, but " 
indicated that none of the studies which he reviewed provided'any test of this 
hypothesis. 

in this study, a temporal ordering of the four most important" background 
varlTbles was established. Prior Subject Interest and Reason for Taking the 
Course (represented by percent indicating "General Interest Only") were 
assumed to come first. Since these two variables were essentially uncorrelated 
(r=+,05, not statistically significant), their combined effect was approxi- 
mately the sum of the separate effects of each. Expected Grade and Workload/ 
Difficulty were assumed to come late.r, and so only variance not-already explained 
by the first two variables was attribuii^d, to them. However, as discussed earlier 



j ... Background Characteristics 19 

I 

i 

i _ 

Expected Grade and Workload/Difficulty were negatively correlated with each 

other trf-29, p< .001) even though both were generally positively related 

i " ' 

tojthe evaluation scores. Th is_cQfflpJlca.tes-thetr interpretation since their 

combined cdntributlot-, was greater than the sum of each of them separately." 

i ■ ■ 

\ Inspection of Table 5 indicat.es that, averaged across al' eva^'iation 

sco'^es, 12.5% of the variance in the evaluation 'scores can be explained by 

the Ifour. background variables. This value, if corrected for the four back- 

ground variables included in each regression, would be 11.9%. Comparison with 
\ ' ' ' . 

Tabli 2 indicates that only Group Interaction (which was related to Enrollment) 

and Breadth of Coverage (which was related to Teacher Rank and percent taking 

the course as a Major Requirement) were substantially better predicted by the 

entire set of 16 background variables. Almost half of the variance 'Which was 

explained by the four background variables was directly attributable to Prior 

'Subject Interest [5:3% of 12.5%), while each of the other -variables contributed' 

no more than 20% of the predictable variance (i.e., no more than 2.5% of the 

li2.5%)'. Controlling for the effect of Prior, Subject Interest and Reason for 

Taking the Cours'e had little effect on the variance explained by Workload/ 

Difficulty, but reduced the effect of Expected Grade by nearly one-third. As 

previously noted, the combined effect of , Expected Grade and Workload/Difficulty 

(5.7% of 12.5%) was- greater than the sum of each separately. 



INSERT TABLE 4 ABOUT HERE 
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In i summary: 

**A var-iety-of-cr-lterla all indlcatea that Prior Subject Interest,, Workload/ 
Difficulty, Expected Grade, and Percent Taking Course for General Interest 
Only were the most important background variables in predicting the 
evaluation scores. 

♦♦Averaged across all the ayaluatlon scores, these four background variables 
. explained 12.5% of the variance Cll.9% when corrected). 

**The explanatory model-usedJn this analysis indicated that Prior Subject 
Interest was the most important background variable, accounting for almost 
half the predictable variance. 

♦♦Controlling for the effect of Prior Subject Interest reduced the effect of 
Expected "Grade by one-third, but had little effect on the other background 
va>:iables. 

CONCLUSIONS AND IMPLICATIONS 

The purpose of the study was to describe the relationship between a set of 
students* evaluations of instructional effectiveness and a set of 16 background 
variables describing the student, the course, and the instructor. Individual 
background variables rarely explained as much as 10% of the variance in any of 
the evaluation scores (3 of 175 relationships) and generally did not even 
explain 5%' (18 of 175). Multiple regression indicated that the percentage of 
variance explained in different evaluation scores varied dramatically, ranging 
from 0% to 25%; the average across the 10 evaluation scores was 11.8%. 
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Canonical correlation showed that 16% of the variance in the students' evalua- 
•tions could be explained by the set of background variables, but this estimate 
was known to have a slight positive bias. 

A variety of criteria each suggested that three or perhaps four background 
variables were most important In explaining variance in the evaluation scores: 
Prior Subject Interest, Workload/Difficulty, Expected Grade, and, perhaps. 
Reason for Taking the Course. Depending upon what assumptions were made? these 
four accounted for either all or more than 80%, of the variance attributable to 
the entire set of 16 background variables. An explanatory model based upon 
these four background variables indicated that almost half the predictable 
variance In the evaluation scores could be explained by Prior Subject Interest 
alone. Controlling for the effect of Prior Subject Interest reduced the 
variance attributable to Expected Grade by one-third, but had little effect 
on the other two background variables. * ^ 

In summary^' the relationship between the 16 background variables and the 
set of students' evaluations definitely existed, but tended to be rather small. • 
Different statistical procedures suggested that, on the average, 12% to 14% of 
the variance in students' evaluations could be explained by the entire set of 
background variables. Even If this entire relationship were assumed to be due 
to biases in the students' evaluations, the magnitude of this bias would not 
obviate their usefulness. Many well -accepted psychological tests probably have 
biases as large as this or larger, even if the people being evaluated are not 
as clever at identifying the biases as are university faculty. However, a host 
of arguments suggest that even the relationship which was found should not be 
considered a simple bias in the students' evaluations. 
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At the most simplistic level, critics of students' evaluations have 
suggested that instructors need only give' high grades and demand little work 
of students in order to be favorably evaluated. If this were true, the bias 
would affect each of the evaluation scores in a similar manner. This simplistic 
notion of bias can clearly be rejected. No one background variable was substan- 
tially related to even half the -evaluation scores, and the percentage of 
variance tn' the different evaluation scores which was explained by the set of 
background variables vari'ed dramatically. Furthermore, the evaluation scores 
which would be expected to be most subject to. bias were not. More than twice 
as much variance was explained in Overall Course Rating than in Overall Instruc- 
tor Rating, and four times as much variance was, explained in Learning/Value than 
in .Instructor Enthusiasm. If students' evaluations are biased, the bias is not 
a simple one. > „ ' .' 

A detailed inspection of the background variables most related" to the 
evaluation scores also undermined the speculation that the relationship is 
caused by a simple bias. The Workload/Difficulty variable was related -to the 
evaluation scores in the opposite direction from what would be predicted by a 
bias. Harder", more difficult courses which required more time outside of class 
were rated more favorably. Prior Subject Interest can be better interpreted as 
a variable impacting the quality of education than a bias which is specific to 
students' evaluations. A high' Prior Subject Interest creates a more favorable 
learning environmenf and probably makes it easier to do a more effective job of 
teaching. Furthermore, Marsh and Overall (1978) showed that this variable 
effected faculty self-evaluations of their own teaching as well as the students' 
evaluations. 

.1 ' 
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, The only background variable which could reasonably .be considered to be a 
possible bias to students' evaluations is Expected Grade, and even this fnter- , 
pretation is subject to alternative explanations. First, the effect of Expected 
Grade was reduced by one-third -^controlling for the effect of Prior Subject' • 
Interest. The most plausible explanation, is that higher Prior Subject Interest, 
causes both better grades and a better ^educational experience. Second, t[ie • " 
Expected Grade relationship can only be considered a bias if the higher grades, ,-,- 
reflect ."easy grading" on the part of the teacher. If higher' expected grades • 
reflect actaal student achievement— better students' evaluations are rela'«,ed 
to better student learning— then most researchers would interpret the relation- 
ship to support the^validity of the ratings. . In reality. Expected Grade . 
probabl^ reflects some unknown combination of both "easy grading" and student 
achievement. ^ 

In conclusion, a variety of multivariate techniques suggested that 12% to 
14% of the variance could be predicted by a" set of 16 background variables. 
However, even this small to moderate relationshi; could not.be interpreted -as 
a.simplfe bias in the students' evaluat^ions. No one background variable was 

substantially related to even a majority of the evaluation scores, the per-. 

'centage of variance explained in different evaluation sjs^res varied- dramati- 
cally, and 'the evaluation scores most likely to be subject^ to bias were not 
the ones best explained by the background variables. Of the three background 
variables most clearly related to the students' evaluations, one (Worklpad/. 
Difficulty) was correlated in the, opposite direction from wh^a^woul^ predicted by 
a bias hypothesis, and another could be better interpreted as affecting- quality 
of 'teaching rather than a bias. Only Expected Grade could reasonably be consid- 
ered\ bias, and even this it)terpretation v^as- subject to alternative explana- 
'trbns. . . ' I . f 

•■ • • ■ • .. \ - 
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T A B L £ ONE \^ 
Correlations Between Background/Oemograph i c Variables and Student Evaluation Scores 

(N=511 Class Averages) 



Prior Interest in Subject ( 1-Low. . . 5-High) 
Workl/ad/Di f f icul ty ( 1-L i gh t/EdSy . . . 5-Heavy/Hard ) 
Expected Grade (0-r..,4-A) 

Course Level (1-Lower Division, 2-Upper Division) 

Taking as"«ajor Elective" 

Taking a^ "General Interest Only" 
Overall GPA 

^ Taking as "Major Requirement" 
% Majoring in Division (Social Science) 
% Freshman-Sophomore Students 

Taking as a "General Education Requirement"^ 

Average "Year in School " j ( 1- Freshman ... 5-Grad . ) 

Enrol 1 ment 
% Junior-Senior Students 

Teacher Rank (1-Lecturer. . .4-Fvul 1 Prof.) 

^% Taking Course i n"Hi nor/Re 1 a ted Field" 

4 Multiple r2 \% variance explained) 



L. 1. 
O) 3 
> O 

o o 

nm 
Em 

.21 
. 17 
.16 
. 16 
.07 

-.15 
.45 

-. 12 

-.11 
.11 

' .10 . 
.11 

-.02 
.07 

20. 4 



«a 1. 

L. 4-» 

a> VI 
> c 
o — • 

.20 

+ . 14 
.20 
.14 
.13 
.12 
.07 

-.12 
.14 

-.12 

-.08 
.10 

-.09 
.01 

-.08 
.06 

8.9 



4.12 

oa 

.21 

EH 

.15 

.10 
-.18 

.15 
-.18 
-.17 

.20 
-.143 

.07 
-.10 

.07 
24.7 



4-» fO 



+ .06 
.20 
.12 
.06 
.05 
.07 

-.07 
.03 

-.08 

-.04 
.11 
.pi 
.21 

-.12 
.03 

5.9 



C 4-> 

I O • • t. 

C 'r- t. > O 

V< fO -3 a> "r- O. 

CD fO O 4-» T? O. 

t. N I- C C *0 

O O •"•OS 

-.03 T7291 .09 

.01 -.02 .01 

.01 JJT] Al 
.14 




.03 FTTsI -.06 

^.06 Trm .19 
-.03 ETTZP-. 18 
.04 .05^ .13 
-.10 -.14 -.05 
.02 .02 .04 
0 23.0 3.5 




1- Students each indicated one of'these reasons for talcing the coyrse 

2- Correlations in boxes represent relationships in which a'sjngle Background/Demographic Variable accounts for at least 5% 
*of the variance in a student evaluation score 

3- These relationships show substantial non-linear trends(i.e. quadratic and/or cubic components add at least IX to the 
Variance Explained by the linear relationship^ and Total Variance Explained was at least 5%) 

^4- See Table Two for details on how this value was obtained 
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TABLETWO . . ^ 

Multivariate Relationship Between Each sTudent'Evaluation Score and the Entire Set of Dackground/Domographi c Variables 

/ (N = 511 Class Averages) 



OVERALL COURSE RATIHG 
Step I Variable 

1 Prior Interest 
" 2 Hork/Diff. 

3 Expected Grade 

4 % General Interest 



aR2 

10.4 
3.5 
4.9 
3.4 



Beta 
.73 
.34 
.32 
. 18 



r 

.32 
.23 
.21 
.16 



GROUP INTERACTION 
Step # Variable 

1 Enrol Iment 

2 Expected Grade 

3 Prior Interest 

4 % Majoring, in Div. 



^AR^ 
. 102 
.003 
.025 
.018 



X General Interest .011 



Peta r 
.21 -.32 
.24 .31 
.16 .29 
.18 .29 
.11 .07 



Total Variance Accounted 22.2% (20. 4%) 



OVERALL INSTRUCTOR RATING 
Step # Variable 

1 Prior Interest 

2 Expected Gi^a.de 

3 Work/Drff^. 

4 % General interest 



aR2 

3.9 
. 2.6 

2.9 



Beta 
.12 
.24. 
21 



1.6 .13 



r 

.20 
.20 
. 14 
.12 



Total Variance Accounted 24.7% (23.0%) 

INDIVIDUAL RAPPORf * « 

Step # Variable aR*^ Beta r 

1 Avg. Year In School .036 .11 .19 

2 Enrollment .013 -.12 -.18 

3 Expected Grade .011 .12 .17 



Total Variance Accounted 11.1% (0-9^) . Total Variance Accounted 6.0% .(3.5%) 



LEARNING ^ 

Step # Variable aR^ Beta r 

' 1 Prior Interest . 194 .36 .44 

2 Expected Grade .038 .26 .29" 

3 Work/Diff. .018 .17 .12 

4 % General Interest .021 .15 .15 

Total Variance Accounted 27.1% (24.7%) 

MORGAN I ZAT ION 
Step I Variable 

%X % General I nterest 



BREADTH 
Step i 
1 
2 
3 



OF COVERAGE « 
Varlalile aR*^ Beta 

% Major Requirement .068 -.27 
Work/Diff. .039 .18 

Teacher Rank .029 .18 



r 

-.26 

.15 

.24 



Total Variance Accounted ^ 13.6% (11.3%) 



aR2 Beta r 
.022 . 16^ . 16 



ENTHUSIASM 

Step # Variance 

1 Prior Interest 

2 Expected Grade 



ar2 Beta r 
.050 .20 .23 
.034 .16 .20 



Total Variance Accounted 2.2% (0%T 



Total Variance Accounted ' 8.4% (5.9%) 



EXAMINATIONS - ^ 

Step # Variable 

1 Expected Grade 

'2 Work/Diff. 

3 'ttjacher Rank 

4 PORSR 

5 t General Int. 



aR2 Beta r 
.031 .22 .18 
.024 .20 .10 
.018 -.15 -.14 
.014 .13 .13 
.016 . 13 ■ . 10 



Total Variance Accounted 10.3%(d.3%) 



ASSIGNMENTS 
Stop 9 Variable 



1 Work/Oiff. 

2 % Maj . Requi re, 

3 Expected Grade 

4 % General Int . 



ar2 Beta r 
.051 .33 .23 
.045- -. 13 -. 17 
.038 .21 .13 
.010 .14 .18 



Total Variance Accounted 14.4% J[12.3%) 



WORKLOAD/DIFFICULTY 
Step H Variable 

1 Expected Gra.de 

2 Overall GPA 

•3 Prior Interest 

4 %-flaj. Requi re< 

5 Teacher Rank 



aR2 Beta r 

.087 -.42 -.29 

^ 071 .25 . 12 

.024 .20 .12 

.019^>. 17 . 17 

. .011 .12 .11 



Total Variance Accounted 21.2%Ml9.6%) 



was 



Included-^only if the aR2 (the cha^ in Tpt'al Variance /Accounted resulting from'the Step) was 



4 - 



Each addtional step 

■?S5^rn5.l5?a"rv;riance Accented haVo boen adjusted for thf "-J-'f^-iables Included at each ^step (J^^;/^' ^ J?^,) , „ 

The vklue of "Total Variance Explained" which appears '" -P^"" .''^J.Jf" TteSwI e regression procedure (fohen & , 

in the set of Background/Demographic variables as is appropri a te when using a stepwise regression proc y 

S'aCeJagl'of the total variance estimates i" paren-theses^-n l|^§-Jthe i,ver^ge of ^val ue^.n^t Ea5fSth|&e| 1^^14«)^.n^ 
Srrnilfs!%rrll?l5;S?rfi2SlWrrxJlired°frrirn^td^?a?]in^^ score in determining the aver.^.s. 
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TABLE THREE 



The "Unique" Contribution of Each Background Variable 
(N=511 Cla.ss Averages) 

STUDENT EVALUATION SCORES 



BACKGROUND/DEMOGRAPHIC ' 
VARIABLES 

Prior Subject Interest' 

Workfaad/OUficul tv ' 

Expected Grade ^ 
2 

Reason for Taking Course 
% "General Interest Only" 
t "Major Elective" 
% "Major Requirement** 
% "General Ed Require" 
t "Minor Related Field" 

2 

Year in School/Course Level 
Course Level 
t Frosh-Soph in Class 
X Jr-Sr in Class 
. Avg. "Year in School" 

Overall GPA (prior) 

% Division Majors 

Enrollment 

Teacher Rank 



Overall 
Course 



Overall 
Instructor 



2.7X 


0.62 


6.72 


B.9« 


3.82 


3.62 


5.6)^1 


4.42 


5.02 


3.3X 
(3.'3«) 
(O.IX) 
(2.42) 

(o.oj:) 

(0.4X) 


1.82 
(2.02) 
(0.02) 
(1.62) 
(0.12) 
(0.22) 


3.22' 
(2.32) 
(0.82) 
(2.52) 
(1.62) 
(0.32) 


0.02 
(0.0%) 
(0.02) 
(0.02) 
f0.02) 


0.0*2 
(0.02) 
(0.0) 
(0.02) 
(0.02) 


0.02 
(0.p2) 
(0.02) 
(0.02) 
(0.02) 


1.02 


0.42 


1.02 


0.12 


0.52 


0.02 


0.02 


0.02 


0.02 


0.72 


0.02 


0.62 



* Group 

Learn Ethusiasm Organization Instruction 

1.92 0.1>: 0.92 

1.32 0.22 ' 0.02 

3.12 0.12 3.82 . 



0.22 
(0.82) 
(0.62) 
(0.62 

0.22) 

:o.o?i^ 



0.02 
(0.22J 
(0.12) 
(0.02 
(0.32 

0.32 

0.02 

0.32 

1.02 



3.42 
(3.62) 

(0.02 
2.02 
0.52, 
0.02) 

(1.72| 
(0.62) 
(0.02) 
(0.12) 
(0.02) 

0.02 

0.92 

0.02 

0.92 



1.12 
(1.12) 
(0.02) 
(0.12) 

0.22 
(0.02) 



0.02 
0.12) 
.0.02) 
(0.02) 
(0.02) 

0.02 

1.12 

2.52 

1.02 



I'-.dividual 
Rapport 



0.02 

0.12 

2.02* 

, 0.62 
(0.02J 
(0.02) 
(0.02) 
(0.92) 
(0.02) 

1.52 
(0.02) 
(1.22) 
(0.02) 
(1.32) 

0,02 

0.02 

1.12 

0.02 



\ 

V 

\ 

Sreadth Exams Assignments 
0.82 0.32 3.42 



2.82 



2.12 
0.02 

(1.72) 
(5.42) 
0.02) 
1.02) 



1.32 
(0.22 
(0.02: 
(0.02) 
(0.12) 

0.02 

0.12 

0.02 

1.82 



2.52 
4.52 

1.22 
(1.92) 
(0.02) 

0.52 
(0.02) 
(0.02) 

1.32 
(0.12) 
(0.02 
(0.52 
(0.52) 

0.02 

0.42 

0.22 

1.92 



5.82 



1.92 

3.92 
(3.92) 
(0.12) 
(2.52) 
(0.82) 
'X> (0.32) 

0.22 
(0.02) 
(0.02) 
(0.12 
(0.02) 

0.02 

0.02 

0.22 

0.82 



13.12 



2.12 
(0.52) 
(1.12) 
(*2.32) 
(0.62) 
(0.02) 

1.12 
(0.02) 
(0.12) 
(0.32) 
(0.02) 

3.52 

0.92 

0.02 

0.62 



1 2 

Uniqueness was defined as the adjusted change in R due to the introduction of each 8ackground/0emographic variable after all other variables are 
entered. 

2 • ^ 2 

For these two "sets" of variables, all variables were entered in one step and the change in adjusted R for the entire set of variables is presented. 
Values in parentheses indicate the change in adjusted r2 resulting form each variable separately, nnt considering other variables in the set. The 
contribution of one variable can be as high or higher than for the entire set since the R^ is adjusted for the total number of variables included at 
each step. 
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TABLE FOUR 



Canon ica.l Variable Loadings for 511 Class Averages 
(correlation between canonical variables and original variables) 



Student EvaJuation Factors 

Overall Course ' 

Overall Instructor ^ 

Learning 

Enthusiasm 

Organization 

Sroup Interaction 

Individual Rapport 

Breadth 

Examinations 

Assignments ^ 

£Jri2)/n 

Background Variables 

Prior Interest 
Workload/Difficulty 
Expected Grade 
Course Level 
% Major Elective 
% General Interest Only 
Overall GPA 
% Majdr Required 
% Majoring in Division 
,% Fosh Soph 
'% General Education 
Avg. Year in School 
Enrollment 
% Jr. Sr. 
Teacher Renk 
% Minor/Related Field 
. e(rj2)/n 

Canonical R 
Canonical R^ ^ ' 

Redundancy in Evaluations 
Explained bv Background 
variablesli2 

Redundancy in Background 
variables explained by 
Evaluations 



I 


II 


« III 


IV 


V 








49 


-.11 


AC 

.45 


no 




• JO 


- 02. 


.78 


-.12 


'.17 


.16 


-.23 


.40 


-.13 


.30 


• .07 


-.14 


-.03 


-.15 


.03 


.52 


-.35 


.64 


-.35 


-.34 


.31, 


.17 . 


.24 


-:i7 


-.15 


.02 


.02 


.17 


.54 


-.27 


.49 


-.43 


.22 


-.19 


.01 


.76 


-.01 


.45 


.30 


.06 


.45 


■ -.34 


(.212) 


(.062) 


(.043) 


(.179) 


(.054) 



.79 


-.10 


.27 


-.24 


-.05 


.22 


.49 


.48 


.35 


.29 


.55 


-.34 


-.27 


.01 


.04 


« DD 


09 ' 


- 35 ■ 


-.05 


.09 


.53 


.20 


-.40 


-.07 


-.12 


.15 


.06 


.03 


.32 


-.64 


.29 


-.11 


-.11 


.23 


\-.43 


,-.26 - 


-.32 


.11 


-.08 


, .57 


.'32 


-.05 


-.40 


.32 


.39 


-.38 


.26 


..37 


.03 


-.15 


-.40 


.21 


.31 


-.08 


-.26 


.45 . 


,.-.27 


-.29 


-.29 


.08 


-.31 


.34 ' 


.48 


-.17 


-.25 


.19 


-.15 


-.33 


.31 


.22 


-.02 


.76 


-.41 


-.20 


.04/ 


.13 


.18 


-.04 


,00 


-.16- 


(.156) 


(.092) 


(.014) 


(.044) 


(.089) 


.67 


.61 


.44 


"'.42 


, .32 


.45 


.37 


.19 


.18 


.10 


9.5% 


2.3% 


0.8% 


4 

3.2% 


0.5% 


7.2% 


3.4% 


2.0% 


0.8% , 


0.9% 



14.1% 



^Redundancy is an estimate of the total variance in one set of vaHables (evaluation scores) 
which is explained by another set of variables (background variables). 

2Redundancy is defined as (R2) (^r2/n) for each canonical variate. Total redundancy is the 
sum of* these values for each of the canonical variat.es. 

^he 16.3%,- an estimate of the proportion of variance explained by the background variables, 
is Dositively biased. Although a correction for this bias has riot been developed, the true 
oopSlItion figure would be closer to the 11.8% estimated on the basis of the multiple regres- 



logons. (See Table Two) 




Preliminary Path Analysis: 



"Priar. Subject Interest" 
unpartialed 
^ 2. "X Indicating General Interest* 
unRarti.aled 
1. S 2. in Combination 
unpartialed 
3- "Expected Grade" 

unpartialed 
4. "Workload/Difficulty" 
unpart i aled 

3. S 4. in Combination 
Unpartialed 

3, .with K & 2. 
p'drtialed out 

4. with 1. & 2. 
partialed out 

3. & 4. in Comb, with I. & 2. 
partialed out 

V 

' Total Variance Explained 



TABLE FIVE 

Evaluation Score Variance Explained By Four "Most Important" Background Variables ^ 



Notation^ 



Over 
Crse. 



Over 
inst. 



Lear. Enth. Org . 



Group'lnd. Avg. 
Int. Rap. Brea. E xams Ass ig. Fig. 



2 


1 


10. 


6% 


.4. 


IX 


19. 5X 


5.2X 


0. 


IX 


B. 


3X 


0. 


9X 


0. 


IX 


0. 


IX 


3. 


9X 


5.28X 


2 

ry*" . 


2 


?. 


7% 


1. 


3X 


2.3X 


0.9X 


2. 


4X 


0. 


5X 


0. 


OX 


3. 


BX 


1. 


OX. 


3. 


IX 


1.80X 


R2y. 


1.2 


12. 


7% 


5. 


2X 


21. 2X 


Lll 


2. 


6% 


'8. 


ii 


0, 


9% 


ii 


11 




il 


6. 




6.86X 


R^y. 


3 ' 


4 


6% 


4. IX 


B.3X 


4. IX 


0, 


OX 


9, 


9X 


'3, 


OX 


0. 


OX 


3, 


3X ' 


1, 


6X 


3.89X 


R^y. 


4 


5 


3% 


1, 


BX 


1.4X 


0.4X 


0 


OX 


4 0 


OX 


0 


OX 


2. 


4X 


1 


OX 


5, 


37X 


1.76X 


R^y. 


3.4 




At 


8 


5X 


12. BX 


5.7X 


0 


ox 


11 


9X 


3 


.5X 


2, 


5X 


5 


.9X 


9 


5X 


7.44X 


R^y 


(3.1,2) 


2 


.2X 


2 


7% 


3.9X 


2.5X 


0 


.ox 


7 


.BX 


2 


.5X 


0 


OX 


3 


.2X 


0 


.8X 


'2.56X 


R^y 


.(4.1,2) 


4 


.6X 


1 


.6X 


0.7X 


0.2X 


0 


A% 


0 


.2X 


0 


.2X 


3 


5X 


1 


.3X 


5 


.5X 


1,79X 




.(3,4.1,2) 


10..2X 


6_ 


.6X 


6.5X 


3.Ct 


0 


.1% 


8_ 


.)X 


2 


• BX. 


3 


.BX 


6^ 


,5X 


8 


.6X 


5.68X 


R^y 


.1,2,3,4 


22 




11 


.BX 


27. 7X 


9.5X 


2 


.7X 


16 


.7% 


3 


.7X 


7^ 


. 7X 


/ I 


.5X 


15 


.2X 


iif.54X 



1 Tbe model used assumes that (1) Prior Subject Interest A Reason for'Takiog Course 
\ 



precede (3) Expected Grade and (4) Workload/Difficulty 



2 The notation used is that r^ refers simple correlations R2yl,2 refers to the total variance in each (y) evaluation score which is explained by 
8ackS?o2nHariable^ (1) and (2): Ry. (3,4* 1,2) refers to^the total variance in each evaluation score which is explained by Background variables 
(3) & (4) which is unique from that explained by variables (1) & (2). 

3 None of the variance estimates are'corrected for the number of variables in the regression equation. The Average of Total Variance Explained 
(12.54X) if corrected for 4 Background Variables would be 11.85X and 9.71% if corrected for the entire set o^f 16 variables; 



» .c 

> 
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A p p E r: 0 I X n 



RELIABILITY 



Evluttlon Ittms 



ANOV^ Reliability Est1?:sates For 
Class^ Averages Based Upon 
Different Nunbers of Response^ 



I. LEARNING/VALUE 

Increased Interest as Course Consequencje 
learned Something ^Valuable / 
leimed I Understood Subject Hatter / 
OVEfUUl COURSE RATIKG / 
InttlUctMlly ChalT^glng/Stlvulatlng 

It* EHTHUSIASN 

Qynaiiic ^ Energetic \ — 
Enhanced with Humor \ , 
Held Your Interest \ 
Enthusiastic About Teacning 
' OVERAU INSTRUCTOR RATING 

III.iRGANIZATION \ 
Materials Prepared & Explained 
Instructor Explanations Clear 
Lectures Faciliteted Note Taking 
Objectives Stated and Pursued 

, IV. GROUP IKTERACTION \ 
^ Students Shared Ideas/Knowledge 

Encouraged to. Participate ' \ 
' Encouraged to Express (Vm Ideas \ 

Encouraged to puestion 4 Given Answers 

V- INDIVIDUAL RAPPORT * ' \ - 

Welcwwd Seeking Help/AJvlce \ 
Interesti^dvfn Individual' Students 
Accessible to Students 
'Friendly Toward Studentsr 

VK BREAOTH'OF COVERAGE • 

Presented Background of Copcepts 
Contrasted Implications 
Presented Different Points of View 

« Discussed Current Developments 

VII. E)OWINATIONS 

Evaluation Hethods Fair/Appropriate 
Tested Actual Content 
Exam Feedback Valuable ^ 

viii.'assigwents 

. Readings/Text Valuable 
Contributed to Understanding 

IX. WORKLOAD/DIFFICULTY 
Workload (Light>Heavy) 
i)1fficulty (Easy>Kard) 
Hours Out of Class 
* Pace (Too Slow-Too Fast) 



MEDIAN RELIABILITY 



5 


10 


15 


25 


50 ' 


100 




.69 


.77 


.83 


.91 


.96 


.55 


.71 


.78 


.86 


.92 


.96 


.50 


.67 


.78 


.85' 


.92 


.95 


.62 


.76 


.83 


,89 


.94 


.96 


.64 


.78 


.84 ' 


.90 


.95 


.97 


.70 


.83 


.88 


.92 


.96 


.98 


.69 


.81 


.87 


.92) 


.96 


.98 


.67 


.80 


.86 


.91 . 


.96 


.97 


.66 


.79 


.85 


.91 


.95 


.97 


.66 


.80 


.85 


.91 


.95 


.97 












« 


.58' 


.74 


.81 


.88 


.93 


.97' 


.60 


.75 


.82 


.88 


.94 


.97 


SO 


.75 


.82 


.88 


.94 


.97 


.51 


.68 


.76 


.84 


.91 


.97 














.64 


.78 


.34 


.90 


.95 


.97 


\65 


.79 


.85 


.90 


.95 


.97 


.61. 


.76 


.82 


.89 


.94 


.97 


.60 


,75 


.82 


.88> 


.94 


.97 


.57 


.72 • 


.80 


.87 


.93 


.96 


.57 


.73 


.80 


.87 


.93 


.96 


.52 


.69 


.77 


.85 


.92 


.96 


.57 


.73 


.80 


.87 


.93 


.96 


.55 


.71 


.78 


.86 


.92 


.96 


^.52 


.69 ^ 


.77 


.85 


.92 


.96 


.50 


.67 


.75 


.83 


.91 


.95 


.56 


.71 


.79 


.86 


.94 


.97 


.58 


.74 


.81 


.88 


.93 


.97 


.58 


.74 


.81 




".93 


.97 


.59 


.74 


.81 


.88 


.94 


.97 


.63 


.77 


.84 


.90 


.94 


.97 


.50 


.67 


.75 


.83 


.91 


.95 


.60 


.75 


.82 


.88 


.94 


.97 


.52 


.69 


.77 


.85 


.92 


.97 


.55 


.71 


.78 


.86 


.92 


J.96 


.36 


.52 


.62 


.73 


.85 


-.92 


.58 


.74 


.81 


.88 


.93 


.97 



Coefficient Alpha^ 
Reliability Estimates 
of Factor Scores 

.95 



.97 



.93 



.98 



.95 



.93 



.94 



.90 



.88 



.9i 



•Anova Reliability estimates were obtained by taking 10 .responses from each of 387 courses In which 
at least 15 students responded. A one-way Anova was performed 1n>h1ch the courses served as levels. 
The reliability estimate for 10 responses was confuted by subtracting the reciprocal of the F-Ratio 
fron 1.0. The other estimates were generated with the Speannan-Bro^/n equation. This procedure Is 
described In Winer (1971), t^rsh (1976) and Centra (1973). 

ft 

-Coefficient Alphas We cofnputed with Method^ described by Nie, et. al. (1977). 



Two types of rellat^llity are presented above. The Anova reliability estimates measure the relative consistency 
within each class relative to the differences between different classes. The principle source of error 
measured by this technique Is the, diversity of student opinion within the courses. It should be noted that 
this is a more stringent criteria than would be measured by assessing tbe reliability of Individual responses. 
Using the Spearman-Brown equation, the median reliability for a sample size of one would be r ■ .22. However » 
using a'tcst-retest procedure over a ;tWee year interval. Overall and Harsh (1978) found that reliabilities 
of the responses of Individual students were generally over .50. ' 

• ' The coefficient alpha reliability Is based upon the degree of intercorrelation among the Items defining each 
factor. This valUc will also vary with the number of responses. The average number of responses In the 
Sll courses used In this analysis was 26.7. (Avg. Enrollment was 34.56, Avg. Response Rate was 77%). The 
median reliability of the factor scores is substantially higher than the median reliability of individual 
items based upon a comparable number of responses. Thls^ is due, at least In part, to the greater reliability ' 
of an'average. ^ 
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APPENDIX III 



Ydctor Analysis of Student Evaluatinn Instrunont (>31l Class Average Responses) 
- ^ Factor Pattern Loadings 



11. 



Evaluation Items (par^brased) 



LFAmilNG/VALUE 

Increased Interest as Courso Consequence 

learned Soincthing Valuable 

Learned 4 Understood Subject Hatter 

OVERALL COURSE RATING 

Intellectually Challenging/Stimulating 

INTIIUSIASII 

Dynamic * Energetic * 

Enhanced with hutnor 
Held your Interest 
Enthusiatic about Teaching 
OVERALL INSTRUCTOR RATING 



III. 



ORGANIZATION . \ 
Materials Prepared 4 Explained 
Instructor Explani^ons Clear 
Lectures facil itated Note Taking 
Objectives stated & pursued 



IV. GROUP iriTERACTlOM ** 
Studei)t$ shared Ideas/knowledge 
Utcouroged to Participate 
Encouraged to Express Own Ideas 
Encouraged to Question & Given Answers 

r. INDIVIDUAL RAPPORT 

Welcomed Seeking llelp/Advice 
Interested in Individual Students 
Accessible to Students 
Frierrdly Towards Students 

VI. CREADTIl OF COVERAGE 

Presented Background of Concepts 
Contrasted In«pl Icatlons 
Presrntcd Different Point* of View 
• Discussed Current Dev '.jpnents 

Vn. EXAMINATIONS ^ . ... 

Eval I^thods Fair/Appropriate 
Tested Actual Content 
Exam' Feedback Valuable 



^III. 



IX< 
0 



ASS16i<l4ENTS 

Readings/Text Valuable , 
Contributed to Understanding 

WORKLOAD/DIFFICULTY 
Workload (Light-lleavy) 
Difficulty (Easy-Hard) 
Hours Out of Class 
Pace ("Too Slow-Too Fast) 



^tcan 



3.91 

4.15 

4.01' 

3.83 

3.98 



Standard 
Deviation 



3.72 
3.08 



.3.37 
3.45 
2.61 
3.09/ 



56 
48 
41 
65 
54 



.59 
.54 



.61 
.52 
.61 
.39 



-02 
09 



10 
-02 

13 
•09 



II 



M 
06 

:i 

23 
17 



3.90 


0.65 


08 


67 


3.85 


0.65 


01 


'67 


3.66 


0.67 


14 


65 


4.-18 


^0.57 


10 


48 


3.97 


*\).65 


14 


43 


3.90 


0.56 


12 


-06 


3.90 


0.56 


18* 


12 


3.77 


0.62 


08 


-02 


3.94 


0.53 


2(> 


-10 


4.07 


.59 


08 


10 


4.05 


.60 


11 


12 


4.09 


.55 


06 


12 


4.08 ' 


.55 


09 


13 



4.13 


.54 


08 


10 


4.02 


/ 


06 


18 


3.91 


/ .56 


-02 


08 


4.28 


' .49 


00 


-25 


3.97 


/ .48 


12 


05 


3.94 


' .49 • 


06 


10 


4.63 1 


.44 


03 


08 


4.14 / 


.49 


19 


1^ 


3.80 ( 


.56 


03 


04 


3.88 J 




09 


02 


3.67 


^ .59 


03 


05 



-05 
01 



02 
02 
03 
01 



III 


IV 


V 


VI 


-64 . 


06 


04 


. 09 


\\ 


12 


00 


04 


17 


09 


06 


-09 


12 


07 


05 


07' 


03 


08 


-01 


18 


15 


07 


04 


09 


16. 


00 


08 


06 


26 


06 


02 


03 


19 


07 


13 


14 


25 


10 


17 


11 



-01 
03 
04 
17 



03 
14 
-19 



07 
03 
06 
08 



07 
07 
16 
16 



05 


06 


82 


06 


17 


69 


03 


01 


65 


12 


18 


61 


12 


02 


05 




07 


08 


'17 


12 


03 


15 


16 


02 


02 


05 


16 


10 


02 


06 


09 


10 


16 


02 


00 


04 


06 


10 


02 


02 


08 


00 


00 


-01 


-01 


-10 


01 


10 


11 


-10 


-05 



14 
08 
27 
12 



04 
00 
07 
04 



-02 
-06 
24 
-08 



68 

67. 
00 

05 
09 
-02 



11 
01 



00 
11 
07 
12 



ut t 
VII 


vt It 
fill 

/ * 




06 


17 


00* 


11 


15 


18' 


10 


* 12 


-28 


20 


17 


10 


19 


13 . 


31 


09 


11 


07 


07 


10 


00 


03 


10 


01 


13 


09 


06 


12 


08 • 


05 



14 

12 
08 
24 



07 
00 
11 
12 



10 
14 
11 
10 



07 
01 
12 
17 



-01 
16 



00 
07 
-09 
14 



10 
08 
11 
11 



08 
07 
04 
07 



03 
03 
11 
06 



12 
20 
12 
12 

15 
14 
07 



Od 
08 
12 
04 



04 
-07 
-03- 

04 



00 
00 
00 
-02 



-01 
00 
07 

-09 



-03 
04 
-03 
-06 

-06 
-04 
09 



02 
03 



89 
85 
76 
60 



1— Factor Analysis was Oblique (correlated) with the Delta Factor^-2.0 (Nie, et. ,al , 1976) 
2"First nine eigenvalues were 19.8, 3.3, 2.3. 1.5, -1^2, 1.0. .76, .60^ .50 
O )rrelations between Factors ranged from r'-.Ol to r».49 (Median r'.27) 



items except Workload/Difficulty were answered alono 5-point response scale (I Very^Pgor, ^Mo^^F^ate 5-Very Good). Morkload/Dlff itulty 
tl^^t varied on 5.polnt response scalr> with end-points M»ovr>, excrpt for Hours to 2, 2-2 to 5. 3-5 to 7, 4-8 to u^S-Ovcr 12). 



. . **nhis -is a sample ot the SU^WARY REPORT that is sent to individual faculty and/or Department Clj**fpersons*** 

' * ^^^^^ 

instructor: or- 'DOE J. • course: XXXXXXXX 999 class scm^oule number: 93J»<^'sfeM ester: fall 
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NUM9CR OF STUDENTS COMPLETING EVALUATIONS: 26 



PERCENTAGE OF ENROLi/eo STUOENTSf COMPLETING EVALUAT lONS: 76« 



STUQCNT AND. COURSE CHARACTERISTICS: I ^ . . c % tc ooF^iF^rFn 

FOR EACH OUESTlOr; THE PERCENTAGE OF STUOENTS MAKING EACH RESPONSE ANO TMf MEAN AVERAGE RESPONSE (IF APPROPRIATE! IS PRESENTED, 

NUMBER OF STUDENTS ACTJIALLY RESPONDING TO . THE ITEMl. 



(THeSE S.TATIST'^CS ARE BASED UPON THE 
STUDENTS KHO COMPLETED THE EVALUATION FORM BUT DID NOT RESPOND TO A PARTICULAR QUESTION 
PERCENT AGE. 



tN AOOITIONt THE PERCENTAGE OF 



IS INDICATED BY THE "NO RESPONSE" 



PRIOR. 'INTEREST 




OVERALL 


G.P.A. 


EXPECTED 


GRADE 


REASON IN CLASS 


YEAR IN' SCHOOL 


PCT 






PCT 




PCT 


PCT 


PCT 


t-VERf .3« 


0 




l-BELOW 


2.5 0* 


0-F ^ 


0 


IH4AJ REQRO .40 


l-FRESHMAN 4 




0 




2-2.5 - 


3.0 21 


t-o 


0 


2-MAJ ELECT 26 


2-SOPHOMORE 12 


a-MEDlUM 


40 




3-3.0 


3.4 50 


*2-C 


17 » 


3-GCN ED «E0 4 


3-JUNlOR 16 


4- 


16 




i|.3#4 - 


3.7 t-r 


3-B 


46 


4-MlN/RELTD 8 


« 4-SENlDR 46 


5<-VERY HIGH 


44 




5-ABOvi^ 


3.7 1 3 


4-A 


35 


. Vgen INTRST 20 


5-GRADUATE 20 


S3 ^ESPOSSE 


3 




NO RESPONSE 7 


NO RESPONSE 1 1 


NO RESPONSE 3 


NO RESPONSE 3 



/ 


.3*' • 




^^an: 3 


• 2 




OUTSIDE HRS/WK 




COUPSE DIFFICULTY 


COURSE WORKLOAD 


COURSE PACE 






PCT 


PCT 




PCT 


PCT 




l-'VFRY EASY^-, 
2- 


4 


l-VERY LIGHT 0 


t-^TOO SLOtf 


0 


1-0 TO 2 6 




0 


2- 0 


2- 


0 


2-2 TO 5 35 




3-MEDlUM 


77 


3-MEDlUM 73 


3-RIGHT 


85 


3-5 TO 7 46 




4- 


15 


4- . 15 




12 


4-6 TO 12 12 




5-yCRY HARD 


4 


Y HEAVY t 2 ' 


5-TOO FAST 


• 4 


5-OVER 12 0 




NO* RESPONSE 


0 


NO RESPONSE 0 


NO RESPONSE 


0 


NO RESPONSE 0 




^EAN: 3.2 




* mea'n: 3.4 


rCAN: 3.2 




mean: 2.6 



MAJOfl OEPRTMNT 
PCT 

1- SOC SC4 

2- NAT SCI 

3- HUMNTIES 

4- 6US1NESS 
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MEAN 



''•'"'*'*rHeirsuiMiSr "or" <except the two overall rating itehs» are -eighteo averages of separate evaluation "^[Js ANO have 

AVERAGE (ACROSS . ALL<. use CLASSES . OF 50. THE STANDARD ERROR «SE. IS A HEASU« O^ 

rOO SMALL TO BE 
JPON LESS THAN 

50«' 3^ THE CLASS SHOULD BE INTERPRETED CAUTIOUSLY. THE PERCENTILE RANKS IWHI^H 8"-«N O^t tOO» 

YOUR EVALUATIONS COMPARE -ITH OTHER COURSES IN YOUR COMPARISON GROUP (HIGHER PERCENTILE RANKS AND MORE STARS INCICATE BETTER 
EVACUATIONS*. VOU« COMPARISON GROUP. IS: UNOERGR AOOAT E COURSES NOT TAUGHT BY TEACHING ASSISTANTS. ra,^^* rclaT I Vt TO YOUH 

^ COMPARISCN GROUP ISEE ABOVEl 



AVERAGE (ACROSS ALL<. USC CLASSES! OF 50. THE STANDARD ERROR (SE> IS A MEASURE OF THE RCLIAOlLiTY ur ™ ,r 

SUMMARY ITEMS. IT IS SMALLER (MORE RELIABLE! «»iEN I^ARGER NUMBERS OF STUDENTS ARE =<ESP0NDING ""^"f „ '""^ J 

AGREE«NT AMONG THE STUOENTS COMPLETING THE EVALUATIONS. DIFFERENCES OF LESS THAN ONE STANDARD ERROR ARE Tt 

RELIABLY INTERPRETED. IN GENERAL. EVALUATIONS BASED UPON LESS THAN tO STUDENTS- RESPONSES OR EVALUATIONS BASED U 



E^P 



VALUABLE LEARNING E^PERlENdC. WAS INTELLECTUALLY STIMULATING/CHALLENGING 
INSTR DISPLAYED ENT^IUSIASM, ENERGY. HUMOR C ABILITY TO HOLD INTEREST 



LEA^SI VG 
ENfHUSlAS«< 

0RCANI2MION ORGANl2ATlON/CLAR\ITY OF EXPLANATIONS* COURSE matcriaus. uoju^iivcst •.c^^-.wr, 

GRDJP INTERACT STUDENTS ENCOURAGED TO DISCUSS. PARTICIPATE. SHARE IDEAS C ASK QUESTIONS 

INO^. RAPPORT INSTRUCTOR ACCESIBLE* FRIENDLY, AND INTERESTED IN STUDENTS 

0«?EADTH PRESENTATION C*^ BROAD BACKGRD. CONCEPTS t ALTERNATIVE APPROACHES/THEORIES 

exAMlNATIO^lS STUOEMT PECEPTlONS OF VALUE C FAIRNESS OF EXAMS/GRAOED MATERIALS 

ASSIGNHHNT5 VALUE OF ASSIGNMENTS IN ADDING APPRCCIAT lON/UNDERSTANDI NC TO COURSE 

9VE9ALL COURSE HOW DOES THIS COURSE COMPARE WITH OTHERS AT^U.SoC? (QUESTION 30» 
^'oVERALt: INSTR. HOW POES THIS INSTRUCTOR COMPARE WITH OTHERS AT U.S.C.T (QUESTION'SII 
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use EVALUATION SERVICES 

AS A 'DESCRIPTION OF THrS COURSE/INSTRUCTOR, THIS STATEMENT l§ 

(SELECT THEflEST RESPONSEJC^R EACM OF THF POl LCWiNO sT ATE VENTS L tAv jKt. A «t '>fO^Si, OtANK ONt V if H € '"^Mi Y N< T ^ ^'^^^^ 

;1 IJIARNING: VoufOUNOTH^ course ihfTfeXLecTUAaYCtv^aENOJNQ and stim ^ 

2 ' YOU HAVE LEARNED SOMETHING WHICH YOU CONSIDER VALUABLE 

ST ^' ' ^ ] y YOUR IriTERESr IN THE SUBJECT |^AS tNCREASEO AS A CONSEQUENCE OF THiS COURSE 
4 YOU HAVE LEARNED AND UNDERStOOD THE SUBJECT MATFRJALS tN THIS COURSE 

2 .^^,5, JENTHUSIASM: iNSiRucrof? was ehthusiastic about teaching i^t coufi5>£ 

1 6 instructor was dynamic and energetic in conducting the COURSf 

2 r 7 . , • fN^UCTOR ENHANCED PRESENTAnOrlS WITH THE USE OF HUMOR 

P INSTRUCTORS STYLE OF PfleSENTATiON >iELO YOUR INTEREST OiiRlNG CLASS 

CO ^ 

^1 , ' 9/ORGANIZATION; INSTRUCTORS EXPLANATIONS WERE CLEAR * . , * 
O -|Q COURSE MATERIALS WERE /;£LL PREPARED AND CAREfULL^t FXPLA'NED 

m 'PROPOSED OBJECTIVES AGREED WlTh^'^^^-ocCfUALLY TAUGHT SO VOU KNEW WHBRE COURSE WAS GOING 

> ^2 INSTRUCTOR OAVE lECTuRr-JJ>^-'<A^ '..K'N^. NOTfcS 

]^ 13. GROUP INTER ACTIQNi^^^I^DENTs were encourao&d to participate in class discussions 

' < 14 students V;ERE invited to share their IDEAS AND KNOWLEDGE • * ^ 

oc „ ^ , 

^ . "1 5 students were encouraged to /Csk questions & were given meaningful answers 



'•Ui 

*- 

Ui 
0. 



^ 5 STUDENTS WERE ENCOURAOeD ri ) fcXPRE 1 HEiR OWN IDEA5 ANO-QR QUESTION THE INSTRUCTOR 

47 INDIVIDUAL RAPPORT: instructor was friendly towards individual students 

^ ' instructor wade students f-c^i WELCOME 'N SEEKING HELP 7W*CE tN OR OUTSlDE OF CLASS 

^ 19 ' , INSTRUCTOR HAD A G'EN6tNE INTEREST IN INDIVIDUAL students . * , 

,5 20 'NPTRUCTC" .\Av Al.^»>jArt I > 'U.Cf '''">iHLL ^' ^ Tt ,ot •j'^^ DURiN< » -t^Ht t nOURb ^.*»R AFTER Cl ASS 

O 21 BREADTH: 'NSTRUCTOR contrasted THE IMPLICATIONS OF VARIOUS THEORIES 

^ pp ^^4STR^.H Tf'« f f't -t\'« > ^-iAv - >r< < / : - -.I'J .OtA. O'N^t^'^S 0EVELOPED.IN CLASS 

2 23 ' INSTRUCTOR PRESENTEt) POINTS OF VIEW oTH&r THAN MiS HER OWN WHEN APPROPRIATE' . 

o 

J 24 ,NSTRiiC^'-« ADf O ML. > O'^v 1 . ^A^t NT U V CLOPMCNT > IN THE f tEt D 

' 2 25 EXAWIINATIONS: feedback on examinations oRAOfcO fMTER.ALS was valuable 

m ^ 

D 27 ' EXAMINATIONS/GRADED MATERIALS*! EST^ED COURSE CONTENT AS EMPHASISED BY THE INSTRUCTOR 

g 28 ASSIGNMENTS 

< 29 READINGS HOMEWORK ETC CONTHiBuTEO TO APPRECIATION AND UNDERSTANDING OF SUBJECT^ 

O 30 OVERALL: • ' ' . .. ^ h., ^r ., 

}2 HOV/ DOES THIS INSTRUCTOR COMPARE OTHER INSTRUCTORS YOU HAVE HAD AT USC? 

'|, STUDENT AND COURSE CHARACTERISTICS LEAVE blank if no RESPONSE appuES) 

O 32 COURSE DIFFICULTY RELATIVE TO OTHER COURSES WAS i l>VERY E^^Y^S-MEDtUM 5-VERY HARD) ^ 

^ 33 COU^^Se vV^RkLCAC ^r*^- ^ f -v^i^r.'.nT ^^/^'JMtV SVERYMEAVV 



5 34/*CO^SE PACE WAS (1-TOO SLOW 3-A80UT RIGHT 6 -TOO FAST} 



Z 35 H. .■^r-.. V.^t ^ HtftJ 

2 36 LEVfiL OF irvrrEREST IN The SUBJECT PHiUH iuTHit,a>wftSE (t-VERY LOW OvJ^EOIUM &-VERYHIOJV 

e 07 >^EP/'L' ..ir^ ^> ^ . • ; ; . '^^^'^^L-l 

C 38 EXPECTED ORADE IN THE COURSE {1-F ?-D 4-0 S-A) 



0) 

Z 

^ '40' YEAR IN SCHOOL 1) FRESH 2^ SOPH 3 JR 4 SR 5i GR/^D 

41 t * r ' i'MjM i/f.nn tf.f (. 

SUPPLEMENTAL QUESTIONS 'jst Htsf^oribes StLOW PGR instructor s QUESTION^) 
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use EVALUATION SERVICES 

AS A 'DESCRIPTION OF THrS COURSE/INSTRUCTOR, THIS STATEMENT l§ 

(SELECT THEflEST RESPONSEJC^R EACM OF THF POl LCWiNO sT ATE VENTS L tAv jKt. A «t '>fO^Si, OtANK ONt V if H € '"^Mi Y N< T ^ ^'^^^^ 

;1 IJIARNING: VoufOUNOTH^ course ihfTfeXLecTUAaYCtv^aENOJNQ and stim ^ 

2 ' YOU HAVE LEARNED SOMETHING WHICH YOU CONSIDER VALUABLE 

ST ^' ' ^ ] y YOUR IriTERESr IN THE SUBJECT |^AS tNCREASEO AS A CONSEQUENCE OF THiS COURSE 
4 YOU HAVE LEARNED AND UNDERStOOD THE SUBJECT MATFRJALS tN THIS COURSE 

2 .^^,5, JENTHUSIASM: iNSiRucrof? was ehthusiastic about teaching i^t coufi5>£ 

1 6 instructor was dynamic and energetic in conducting the COURSf 

2 r 7 . , • fN^UCTOR ENHANCED PRESENTAnOrlS WITH THE USE OF HUMOR 

P INSTRUCTORS STYLE OF PfleSENTATiON >iELO YOUR INTEREST OiiRlNG CLASS 

CO ^ 

^1 , ' 9/ORGANIZATION; INSTRUCTORS EXPLANATIONS WERE CLEAR * . , * 
O -|Q COURSE MATERIALS WERE /;£LL PREPARED AND CAREfULL^t FXPLA'NED 

m 'PROPOSED OBJECTIVES AGREED WlTh^'^^^-ocCfUALLY TAUGHT SO VOU KNEW WHBRE COURSE WAS GOING 

> ^2 INSTRUCTOR OAVE lECTuRr-JJ>^-'<A^ '..K'N^. NOTfcS 

]^ 13. GROUP INTER ACTIQNi^^^I^DENTs were encourao&d to participate in class discussions 

' < 14 students V;ERE invited to share their IDEAS AND KNOWLEDGE • * ^ 

oc „ ^ , 

^ . "1 5 students were encouraged to /Csk questions & were given meaningful answers 
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^ 5 STUDENTS WERE ENCOURAOeD ri ) fcXPRE 1 HEiR OWN IDEA5 ANO-QR QUESTION THE INSTRUCTOR 

47 INDIVIDUAL RAPPORT: instructor was friendly towards individual students 

^ ' instructor wade students f-c^i WELCOME 'N SEEKING HELP 7W*CE tN OR OUTSlDE OF CLASS 

^ 19 ' , INSTRUCTOR HAD A G'EN6tNE INTEREST IN INDIVIDUAL students . * , 

,5 20 'NPTRUCTC" .\Av Al.^»>jArt I > 'U.Cf '''">iHLL ^' ^ Tt ,ot •j'^^ DURiN< » -t^Ht t nOURb ^.*»R AFTER Cl ASS 

O 21 BREADTH: 'NSTRUCTOR contrasted THE IMPLICATIONS OF VARIOUS THEORIES 

^ pp ^^4STR^.H Tf'« f f't -t\'« > ^-iAv - >r< < / : - -.I'J .OtA. O'N^t^'^S 0EVELOPED.IN CLASS 

2 23 ' INSTRUCTOR PRESENTEt) POINTS OF VIEW oTH&r THAN MiS HER OWN WHEN APPROPRIATE' . 

o 

J 24 ,NSTRiiC^'-« ADf O ML. > O'^v 1 . ^A^t NT U V CLOPMCNT > IN THE f tEt D 

' 2 25 EXAWIINATIONS: feedback on examinations oRAOfcO fMTER.ALS was valuable 

m ^ 

D 27 ' EXAMINATIONS/GRADED MATERIALS*! EST^ED COURSE CONTENT AS EMPHASISED BY THE INSTRUCTOR 

g 28 ASSIGNMENTS 

< 29 READINGS HOMEWORK ETC CONTHiBuTEO TO APPRECIATION AND UNDERSTANDING OF SUBJECT^ 

O 30 OVERALL: • ' ' . .. ^ h., ^r ., 

}2 HOV/ DOES THIS INSTRUCTOR COMPARE OTHER INSTRUCTORS YOU HAVE HAD AT USC? 

'|, STUDENT AND COURSE CHARACTERISTICS LEAVE blank if no RESPONSE appuES) 

O 32 COURSE DIFFICULTY RELATIVE TO OTHER COURSES WAS i l>VERY E^^Y^S-MEDtUM 5-VERY HARD) ^ 

^ 33 COU^^Se vV^RkLCAC ^r*^- ^ f -v^i^r.'.nT ^^/^'JMtV SVERYMEAVV 



5 34/*CO^SE PACE WAS (1-TOO SLOW 3-A80UT RIGHT 6 -TOO FAST} 



Z 35 H. .■^r-.. V.^t ^ HtftJ 

2 36 LEVfiL OF irvrrEREST IN The SUBJECT PHiUH iuTHit,a>wftSE (t-VERY LOW OvJ^EOIUM &-VERYHIOJV 

e 07 >^EP/'L' ..ir^ ^> ^ . • ; ; . '^^^'^^L-l 

C 38 EXPECTED ORADE IN THE COURSE {1-F ?-D 4-0 S-A) 



0) 

Z 

^ '40' YEAR IN SCHOOL 1) FRESH 2^ SOPH 3 JR 4 SR 5i GR/^D 

41 t * r ' i'MjM i/f.nn tf.f (. 

SUPPLEMENTAL QUESTIONS 'jst Htsf^oribes StLOW PGR instructor s QUESTION^) 
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JMST^HJGTOR^'NAME 


DcPARTMENT NAMt 


rnriR^ WTI M RPR" 




INSTRUCTIONS 





This evaluation form \o intended to measure your fH,iv.t.unb t j ihi.s jn^trLi*..tui and course Results will be reported 
f^\o the Department Chairmen to be used part uT/The uvt^aii ^'vaUidiK n ^-f tht^ insiruetor Jhese evaluations will 
have budgetary and promotion<-il imphcatjons s^j ot^^^^** rarx^^ t vOi, senou^l^ When you have finjShed a desig- 
nated student wiH pick up the evaiuationb ana take tht ifx- Dopaf 'ji^^^nt Cnatrperson Your resporises will 
remamanonymousandsumrnanrsv^'iiinut bogivi^ntwU^ uistrvu.t. \ until aftti ihe final grades have been assigned. 
* » * 

• , **'^*Put Instructors Name Departnient Name and CourbO Number at top 

(le Smith Rsychology 200 

Use a number 2 pencH do not u^e »nK 'baa pomt magic marker etc 
Bfacken only one resp<)noe fc'-r each question ana erase any changes completely 



open-ended coMME^nrs ' 



>tfeASttt«OlCAI£ THE IWP0RI4MT CH*RACt£RiC»TfCS Of into iN^Tht^i^ijft .ovS.^t »Vi^«"d/t ot^sVbbi vAwA^lt aft»iNff»a UP'£ft>tbC£t 
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